By Authority Of 

THE UNITED STATES OF AMERICA 

Legally Binding Document 

By the Authority Vested By Part 5 of the United States Code § 552(a) and 
Part 1 of the Code of Regulations § 51 the attached document has been duly 
INCORPORATED BY REFERENCE and shall be considered legally 
binding upon all citizens and residents of the United States of America. 
HEED THIS NOTICE : Criminal penalties may apply for noncompliance. 


This Document Posted By 
Public.Resource.Org, Inc., 
a California Nonprofit 
Organization. 


Not Affiliated Or 
Authorized by ASTM 
or by the United States 
Government 


Document Name: ASTM D2777: Standard Practice for Determination of 

Precision and Bias of Applicable Test Methods of 

CFR Section(s): 


Committee D-19 on Water 
46 CFR 162.050-15(f)(l) 


Standards Body: American Society for Testing and Materials 




% 


O fficial Incorporator : 

The Executive Director 

OFFICE OF THE FEDERAL REGISTER 
WASHINGTON, D.C. 


Q3A^ 











































Designation: D 2777-98 


Standard Practice for 

Determination of Precision and Bias of Applicable Test 
Methods of Committee D-19 on Water 1 

This standard is issued under the fixed designation D 2777; the number immediately following the designation indicates the year of 
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A 
superscript epsilon (e) indicates an editorial change since the last revision or reapproval. 


1. Scope 

1.1 This practice establishes uniform standards for estimat¬ 
ing and expressing the precision and bias of applicable test 
methods for Committee D-19 on Water. 

1.2 Except as specified in 1.3, 1.4, and 1.5, this practice 
requires the task group proposing a new test method to carry 
out a collaborative study from which statements for precision 
(overall and single-operator standard deviation estimates) and 
bias can be developed. This practice provides general guidance 
to task groups in planning and conducting such determinations 
of precision and bias. 

1.3 If a full-scale collaborative study is not technically 
feasible, due to the nature of the test method or instability of 
samples, the largest feasible scaled-down collaborative study 
shall be conducted to provide the best possible limited basis for 
estimating the overall and single-operator standard deviations. 

1.3.1 Examples of acceptable scaled-down studies are the 

local-area studies conducted by Subcommittee D19.24 on 
microbiological methods because of inherent sample instabil¬ 
ity. These studies involve six or more completely independent 
local-area analysts who can begin analysis of uniform samples, 
at an agreed upon time. - . ; . 

1.3.2 If uniform samples are not feasible under any circum¬ 
stances, a statement of single-operator precision will meet the 
requirements of this practice. Whenever possible, this state¬ 
ment should be developed from data generated by independent 
multiple operators, each doing replicate analyses on indepen¬ 
dent samples of a specific matrix type, which generally fall 
within specified concentration ranges (see 7,2.5.2( 3)). 

1.3.3 This practice is not applicable to methodology involv¬ 
ing continuous sampling or measurement, or both, of specific 
constituents and properties. 

1.3.4 This practice is also not applicable to open-channel 

flow measurements. ' ~ . ? ~ 

1.4 A collaborative study that satisfied the requirements of 
the version of this practice in force when the study was 
conducted will continue to be considered an adequate basis for 
the precision and bias statement required in each test method. 


1 This practice is under the jurisdiction, of ASTM Committee D-19 on Water and 
is the direct responsibility of Subcommittee D19.02 on General Specifications, 
Technical Resources; and Statistical Methods. 

Current edition approved,Jan. 10, 1998. Published October 1998. Originally 
published as D 2777 - 69 T. Last previous edition D 2777-96. 


If the study does not satisfy the current minimum requirements 
for a collaborative study, a statement listing the study’s 
deficiencies and a reference to this paragraph shall be included 
in the precision and bias statement as the basis for an 
exemption from the current requirements. 

1.5 This paragraph relates to special exemptions not clearly 
acceptable under 1.3 or 1.4. With the approval of Committee 
D-19 on the recommendation of the Results Advisor and the 
Technical Operations Section of the Executive Subcommittee 
of Committee D-19, a statement giving a compelling reason 
why compliance with all or specific points of this practice 
cannot be achieved will meet both ASTM requirements (l) 2 
and the related requirements of this practice. Precision and bias 
statements authorized by this paragraph shall include the date 
of approval by Committee D-19. 

1.6 In principle, all test methods are covered by this 
practice. 

1.7 In Section 11 this practice shows exemplary precision 
and bias statement formats for: (1) test methods yielding a 
numerical measure, (2) test methods yielding a non-numerical 
report of success or failure based on criteria specified in the 
procedure, and (2) test methods specifying that procedures in 
another ASTM test method are to be used with only insignifi¬ 
cant modifications. 

1.8 All studies, even those exempt from some requirements 
under 1.3 or 1.5, shall receive approval from the Results 
Advisor before being conducted (see Section 8) and after 
completion (see Section 12). 

2. Referenced Documents 

2.1 ASTM Standards: 

D 1129 Terminology Relating to Water 3 

D 1141 Specification for Substitute Ocean Water 3 

D 1193 Specification for Reagent Water 3 

D4375 Terminology for Basic Statistics in Committee 
D-19 on Water 3 

D 5790 Test Method for Measurement of Purgeable Organic 
Compounds in Water by Capillary Column Gas 
Chromatography/Mass Spectrometry 4 

D5905 Specification for Substitute Wastewater 3 


2 The boldface numbers in parentheses refer to the list of standards at the end of 
this practice. 

3 Annual Book of ASTM Standards , Vol 11.01. 

4 Annual Book of ASTM Standards , Vol 11.02. 
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E 177 Practice for Use of the Terms Precision and Bias in 
ASTM Test Methods 5 

E 178 Practice for Dealing with Outlying Observations 5 

E 456 Terminology Relating to Quality and Statistics 5 

E 1169 Guide for Conducting Ruggedness Tests 5 

3. Terminology 

3.1 Definitions —For definitions of terms used in this prac¬ 
tice, refer to Terminologies D 1129, D4375 and E456, and 
Practice E 177. 

3.2 Definitions of Terms Specific to This Standard: 

3.2.1 accuracy —a measure of the degree of conformity of 
a single test result generated by a specific procedure to the 
assumed or accepted true value and includes both precision and 
bias. 

3.2.2 bias —the persistent positive or negative deviation of 
the average value of a test method from the assumed or 
accepted true value. 

3.2.3 laboratory —a single and completely independent ana¬ 
lytical system with its own specific apparatus, source of 
reagents, set of internal standard operating procedures, etc. 
Different laboratories will differ from each other in all of these 
aspects, regardless of how physically or organizationally close 
they may be to each other. 

3.2.4 operator —usually the individual analyst within each 
laboratory who performs the test method throughout the 
collaborative study. However, for complicated test methods, 
the operator may be a team of individuals, each performing a 
specific function throughout the study. 

3.2.5 precision —the degree of agreement of repeated mea¬ 
surements of the same property, expressed in terms of disper¬ 
sion of test results about the arithmetical mean result obtained 
by repetitive testing of a homogeneous sample under specified 
conditions. The precision of a test method is expressed 
quantitatively as the standard deviation computed from the 
results of a series of controlled determinations. 

4, Summary of Practice 

4.1 After the task group has assured itself that the test 
method has had all preliminary evaluation work completed, it 
should prepare the test method write-up in final form. The plan 
for collaborative study is developed in accordance with this 
practice and submitted along with the test method write-up to 
the Results Advisor for concurrence except as specified in 1.3, 
1.4, and 1.5. Upon receipt of concurrence, the collaborative test 
is conducted, data analyzed, and precision and bias statements 
formulated by the task group. The final precision and bias 
statistics must be based on retained data from at least six 
independent laboratories. The statements, with backup data 
including the reported results summary, the calculations lead¬ 
ing up to the statements, and the test method write-up with 
precision and bias statements included are submitted to the 
subcommittee vice-chairman who in turn sends a copy of it to 
the Results Advisor for concurrence before balloting. This 
assures having an acceptable copy of the collaborative study 
results to send to ASTM for items on the main committee 


5 Annual Book of ASTM Standards , Vol 14.02. 


ballot. In most instances, the collaborative study shall be 
complete before a subcommittee ballot. If the collaborative 
study is not complete, the test method may go on the ballot as 
a provisional test method rather than a standard test method. 
Copies of the test data, approved calculations, and statistical 
results shall be filed at ASTM Headquarters when the test 
method is submitted by the subcommittee chairman as an item 
for the main committee ballot. 

4.1.1 The appendix shows an example of “Form 
A—Approval of Plans for Interlaboratory Testing,” as Fig. 
Xl.l. 

4.1.2 For an example of a data reporting form, see Fig. 
X2.1. 

4.1.3 In addition, the appendix shows a sample calculation 
of precision and bias from real collaborative test data, the 
related table of statistics, and the related precision and bias 
statement. 

5. Significance and Use 

5.1 Following this practice should result in precision and 
bias statements which can be achieved by any laboratory 
properly using the test method studied. These precision and 
bias statements provide the basis for generic limits for use in 
the Quality Control section of the test method. 

5.2 The method specifies the media for which the test 
method is appropriate. The collaborative test corroborates the 
write-up within the limitations of the test design. An extensive 
test can only use representative media so that universal 
applicability cannot be implied from the results. 

5.3 The fundamental assumption of the collaborative study 
is that the media tested, the concentrations tested, and the 
participating laboratories are a representative and fair evalua¬ 
tion of the scope and applicability of the test method as written. 

6. Preliminary Studies 

6.1 Considerable pilot work on a test method must precede 
the determination of its precision and bias (2,3). This pilot 
work should explore such variables as preservation require¬ 
ments, reaction time, concentration of reagents, interferences, 
calibration, and sample size. Potentially significant factors 
must be investigated and controlled in the written test method 
in advance of the collaborative test. Also, disregard of such 
factors may introduce so much variation among operators that 
results are misleading or inconclusive (4) (see 9.3 and 9.4). A 
ruggedness study conducted in a single laboratory is particu¬ 
larly useful for such investigations and should be conducted to 
prove a test method is ready for interlaboratory testing (see 
Guide E 1169 for details). 

6.2 Only after a proposed test method has been tried, 
proved, and reduced to unequivocal written form should a 
determination of its precision and bias be attempted. 

7. Planning the Collaborative Test 

7.1 Based upon the task group’s knowledge of a test method 
and having the unequivocal write-up, several factors must be 
considered in planning the collaborative test to properly assess 
the precision of the test method. The testing variables that must 
be considered in planning are discussed below. It is generally 
not acceptable to control significant sources of variability in the 
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collaborative study which cannot be controlled in routine use 
of the test method, because this leads to false estimates of the 
test method precision and bias. In addition, the task group must 
determine within the resources available how to best estimate 
the bias of the test method. 

7.2 Testing Variables : 

7.2.1 It is desirable to develop a statement of precision of a 
test method that indicates the contribution to overall variation 
of selected causes such as laboratory, operator, sample matrix, 
analyte concentration, and other factors that may or have been 
shown to have strong effects on the results. Since any test 
method can be tried in only a limited number of applications, 
the standard deviation calculated from the results of a study can 
be only an estimate of the universe standard deviation. For this 
reason, the symbol s (sample standard deviation) is used 
herein. The precision estimates generated from the study data 
will usually be the overall standard deviation ( s T ) and the 
pooled single-operator standard deviation ( s a ) for each sample 
matrix and concentration studied. 

7.2.2 Laboratories, operators, sample matrices, and analyte 
concentrations are the only sources of variability represented in 
the precision and bias statements resulting from the usual 
collaborative study. They may not represent the additional 
influence that can arise from differences in sample splitting, 
field preservation, transportation, etc., ail of which may influ¬ 
ence routine analytical results as shown in the general precision 
definitions in Terminology D 1129. 

7.2.3 Laboratories —The final precision and bias statistics 
for each analyte, matrix, and concentration must be based on 
data from at least six laboratories that passed all of the outlier 
tests (see 10.3 and 10.4), that is, retained data. To be assured of 
meeting this requirement, it is recommended that usable data 
be obtained from a minimum of eight independent laboratories. 
To guarantee eight providing usable data, it will often be 
necessary to get ten or more laboratories to agree to participate, 
because some may not provide data and others may not provide 
usable data. Maximizing the number of participating laborato¬ 
ries is often the most important thing that can be done to 
guarantee a successful study. 

7.2.4 Even if the single-operator standard deviation is the 
only statistic to be estimated in the study (see 1.3.2), there 
should be a minimum of eight operators providing usable data, 
so you are assured of data from six operators after all outlier 
removal. 

7.2.5 Sample Matrices —The collaborative study shall be 
conducted with at least one representative sample matrix, 
which should be reproducible by subsequent user-laboratories 
so that they can compare their results with the results of the 
collaborative study. 

7.2.5.1 Typically, a reagent water prepared according to 
Specification D 1193 or a synthetic medium, such as the 
substitute wastewater described in Specification D 5905 or the 
substitute ocean water described in Specification D 1141, is 
used as the reference matrix. Analytes may be supplied 
separately as concentrates for addition to this matrix by each 
laboratory or the reference matrix containing the analyte(s) 
may be supplied to each-participant. Information on how the 
reference matrix was prepared in the study shall be clear in the 


precision and bias statement of the test method so users can 
reproduce it properly. 

12.52 Additional collaborative testing should also be con¬ 
ducted using other matrices specified in the scope of the test 
method. Since these matrices must be the same for each study 
participant, they may have to be prepared (or obtained from a 
single source), preserved, and distributed to all laboratories. As 
with the reference matrix, analytes may be supplied in a 
separate spiking solution or already added to the matrix. A 
particularly attractive matrix might be a standard material 
available from an organization such as the National Institute of 
Standards and Technology (NIST). Use of uniform sample 
matrices is necessary in these studies since they enable a more 
certain comparison with the reference matrix than is possibly 
with matrices supplied separately by each participant. 

(1) Use of matrices with naturally occurring, non-zero 
background levels of the analyte(s) being studied will result in 
precision and bias estimates that will be much more difficult to 
properly compare with estimates from the reference matrix. 

(2) Any matrix spiking that may be necessary shall not 
significantly change the natural characteristics of the matrix. 

(3) With the exception of the kind of limited study described 
in 1.3.2, the matrix-of-choice approach, in which each partici¬ 
pant is expected to acquire their own sample of a designated 
type, should not be used. Such studies are basically incompat¬ 
ible with the statistical approaches employed in this practice; 
both the ranking test and the individual outlier test are 
incapable of distinguishing laboratory effects from matrix 
effects. In addition, the presence of variable background 
concentrations prevents the assignment of a proper mean 
concentration level to each precision estimate produced in the 
study. 

7.2.5.3 The same study design should be used for all sample 
matrices. A separate precision and bias statement should be 
generated for each sample matrix with a brief description of the 
matrix tested. 

7.2.5.4 When studies are available indicating the applicabil¬ 
ity of the test method for matrices untested in 7.2.5.1 and 

7.2.5.2 and not meeting the other requirements of this practice, 
at the discretion of the task group responsible for the test 
method and the Results Advisor, and providing the data are 
analyzed in accordance with Section 10 of this practice, this 
supporting data may be included in a separate section of the 
precision and bias statement. A clear but brief description of 
the matrices shall be included and the study protocol em¬ 
ployed. It is the intent of this practice that ultimately, data 
concerning the precision and bias of the test method in the full 
range of matrices covered in the scope and analyzed in 
accordance with this practice, will be made available to the 
users of the test method. 

7.2.6 Analyte Concentrations —If pilot work has shown that 
precision is linear with increasing analyte concentrations, at 
least three Youden pairs (5), that is, six concentrations, 
covering the range of the test method should be included for 
each matrix. If the pilot work suggests that precision is other 
than constant or linear, more concentration levels should be 
analyzed. The study concentrations should generally be rather 
uniformly distributed over the range of the test method. 
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7.2.6.1 Study samples with concentrations at or near the 
detection limit of a test method are likely to produce non- 
quantitative results from many of the participating laboratories 
if participants are permitted to use their detection limit to 
censor their results. Zeroes or less thans that result from this 
censoring process are non-quantitative results and cannot be 
included in the statistical analysis of study results specified 
later in this practice. Conducting the specified statistical 
analysis on whatever quantitative data are available under such 
circumstances can produce misleading precision and bias 
estimates. If it is considered necessary to include samples at or 
near the detection limit, such samples shall be in addition to the 
minimum required three Youden pairs at concentrations that 
can be readily measured by qualified laboratories. Data from 
analyses of the basic three or more Youden pairs that can be 
quantified can then be statistically analyzed as specified to 
produce a proper traditional precision and bias statement for 
the test method. Results from analyses of Youden pairs at or 
near the detection limit can be included in this traditional 
statistical analysis if it turns out that most laboratories report 
quantified results. Otherwise, results for low-level samples 
must be statistically analyzed using specialized procedures, for 
example, procedures similar to those under development in 
Subcommittee D 19.02, which are beyond the scope of this 
practice, 

7.2.7 Since the order of analyses should not be a source of 
systematic variability in the study, each participant should 
either be told to randomize the order of study sample analyses 
or be given a specific random order for their analyses. 

7.2.7.1 Whenever the time of analyses has been shown to 
influence the analytical results, close control over the time of 
analyses will be essential. 

7.2.8 If pilot work has shown that the sample container must 
be of a specific material prepared in a specific manner prior to 
use, the variation in containers obtained and prepared by the 
participants will be a random variable and should be treated as 
such in the planning of the study and in the statistical analysis 
of the data. 

7.2.9 The manner of preservation or other treatment of the 
sample prior to typical use of the test method, if known to 
affect the precision or bias, or both, of results, shall be 
incorporated into the collaborative study design. 

7.3 Measurement of Precision: 

7.3.1 Every interlaboratory study done to provide precision 
and bias estimates for a D-19 test method must use a 
Youden-pair design rather than a replicate sample design. 
Justifiable exceptions to this requirement shall be approved 
through the. process provided in 1.5. In a Youden-pair design, 
each participant receives (or prepares from a concentrate) a 
separate sample for each analysis required in the study. There 
are no replicate analyses; each participant analyzes each study 
sample once and only once, per analyte if appropriate. Among 
the set of samples each laboratory analyzes for a specific 
matrix, there are pairs of . samples containing similar but 
usually different analyte concentrations that differ from each 
other by up to 20 %. As a matter of convenience to whomever 
is preparing, thej samples or spiking concentrates, up to half the 
Youden pairs may. have the same concentration, that is, be 


blind duplicates, but the participants must have no basis for 
comparing their single test results from analyses of different 
study samples. 

7.3.2 The only difference in treatment of data from a 
Youden-pair study is the calculation used to estimate the means 
and standard deviations; these calculations may be found in 
Youden and Steiner (6). Once developed, these mean and 
standard deviation estimates are treated the same as statistics 
from a study with the usual replicate design. A detailed 
example with and without raw experimental data is given in 
Refs. (7) and (8), respectively. 

7.3.3 The value of the nonreplicate design is that the 
single-operator standard deviation estimates are free of any 
conscious or unconscious analyst bias. The procedures for 
calculating overall and single-operator standard deviations are 
given in 10.4 and 10.5 and illustrated in Appendix X3. 

7.4 Measurement of Bias : 

7.4.1 The concept of accuracy comprises both precision and 
bias (see Terminology D 1129 and Practice E 177). As dis¬ 
cussed in Practice E 177, there is not a single form for 
statements of accuracy that can be universally recommended. 
Since the accuracy of a measurement process is affected by 
both random and systematic sources of error, measures of both 
kinds of error are needed. The standard deviation is a universal 
measure of random sources of error (or precision). Bias is a 
measure of the systematic errors of a test method. 

7.4.2 A collaborative study evaluation of bias for a specific 
matrix produces a set of analyte/sample means. The difference 
between a true value (however defined) and the related mean is 
an estimate of the average systematic error, that is, bias of the 
test method. 

7.4.3 There are three major approaches commonly used to 
test a measurement procedure: (7) measurement of known 
materials, (2) comparison with other measurement procedures, 
and (J) comparison with modifications of the procedure itself 
(9). The third approach may involve the standard addition 
technique or the simultaneous analysis of several aliquots of 
different sizes (for example, 0.5, 1, 1.5, 2, 2.5 units). The task 
group will select the approach that best suits its needs within 
the resources available to it. 

7.4.4 The most likely task group approach will be the use of 
known materials. Since reference standards are unlikely to be 
available, the task group will prepare its samples with added 
(therefore known to them) quantities of the constituent(s) being 
tested. The best available chemical and analytical techniques 
for preparing, stabilizing, if necessary, storing and shipping the 
prepared samples should be known within the task group and 
will not be addressed in this practice. However, if the sample 
preparation and handling techniques used for the study are 
different from those expected to be used for samples during 
routine application of the test method, those differences shall 
be pointed out in the precision and bias statement. Future users 
of the test method may decide that these differences had an 
effect on the precision or bias results, or both, from the study. 

7.5 Quality Control During the Study : 

7.5.1 The Quality Control section to appear in the test 
method must be drafted before the collaborative study design is 
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finalized and the study design must assure that the collabora¬ 
tive study will produce any background data not otherwise 
available to properly complete the final Quality Control sec¬ 
tion. Each part of the draft Quality Control section must be 
used during the collaborative study unless insufficient back¬ 
ground data exist to establish credible interim required perfor¬ 
mance criteria for that part. 

7.5.2 All quality control data/information produced to meet 
the requirements of 7.5.1 shall be reported to the task group 
chair along with results from analyses on the study samples. 

8. Collaborative Study Design Approval 

8.1 After approval by the task group, the task group chair 
(or designee) will summarize the proposed design of the 
collaborative study. This summary will include: (7) the test 
method to be tested in ASTM format and as approved by the 
task group; ( 2) the analytes to be included in the study; (5) the 
number of samples in accordance with the paired-sample plan 
of 7.3.1; ( 4 ) the approach for determining the bias of the test 
method as exemplified in the collaborative study; ( 5) the range 
of concentration covered, and approximate concentration of 
material in each sample or set; ( 6) the approximate number of 
laboratories and analysts; (7) the matrices and QC samples 
being tested; (<§) plans for developing study samples; and (9) a 
copy of the instruction and data reporting package to be given 
to each study participant. This summary should be presented to 
the Results Advisor in the form of a letter. 

8.1.1 As an aid, the task group chairman may use, “Ap¬ 
proval of Plans for Interlaboratory Testing,” Form A, and in 
Appendix XI (a completed example is shown in Fig. Xl.l). 

8.2 Upon review of the plan, the Results Advisor will advise 
the task group chairman whether the plan meets the require¬ 
ments of this practice or what changes are necessary to meet 
the requirements of this practice. 

8.3 Upon receipt of approval of the collaborative test plan 
by the Results Advisor, file task group chairman (or designee) 
will conduct the collaborative test. 

9. Conducting the Collaborative Study 

9.1 A single entity, acting for the task group, will prepare 
the samples for the collaborative study and ship them to the 
participants with instructions for the study, a copy of the exact 
test method (if not already supplied), and the participant 
reporting form (or reporting instructions). 

9.1.1 The instructions for the collaborative study shall 
require sufficient preliminary work by potential collaborators 
to adequately familiarize them with the test method prior to 
study measurements. This is necessary to ensure that each 
collaborative study is made by a peer group and that a learning 
experience is not included in the statistics of the collaborative 
study. The task group may also develop procedures to qualify 
prospective collaborators, and this approach is strongly recom¬ 
mended. 

9.1.2 Each laboratory should usually supply its own cali¬ 
bration materials, as independent calibration materials are a 
significant source of interlaboratory variability. However, if the 
cost of availability of calibration materials is judged to be a 
significant deterrent to participation or if currently available 
materials are inadequate and not considered typical for subse¬ 


quent routine use of the test method, these materials may be 
distributed with the study samples. If calibration standards are 
provided, the Precision and Bias section of the test method 
should so note, including the concentrations and matrix of the 
standards and any specific instructions for their use. 

9.1.3 As an aid, the task group chairman may use Form B, 
“Data Report from Individual Laboratories,” as in Appendix 
X2 (a completed example is shown in Fig. X2.1). 

9.2 The batch of samples containing a specific member of a 
Youden pair should be clearly marked with a common unique 
code, informative to the distributors but not informative to the 
study participants. Samples should be sized to supply more 
than the minimum amount necessary to participate in the study 
(with reasonable allowance for pipetting, rinsing, etc.) to allow 
for trial runs and analytical restarts that may be necessary. A 
separate set of samples shall be provided for each operator. 
Sample concentrations should not be easily surmised values (1, 
5, etc.). The assignment of samples to the participating 
laboratories should be randomized within each concentration 
level. The above recommendations should help assure statisti¬ 
cal independence of results. 

9.3 A copy of the test method under investigation, the 
written instructions for carrying out his/her part of the pro¬ 
gram, and the necessary study samples should be supplied to 
each operator. No supplementary instructions or explanations 
such as by telephone or from a task group member within a 
cooperating laboratory should be supplied to one participant if 
not to all. Study materials should be distributed from one 
location, and the operator’s reports should be returned to one 
location. 

9.4 The written instructions should cover such items as: (7) 
directives for storing and subdividing the sample; (2) prepara¬ 
tion of sample prior to using the test method; (3) order of 
analyses of samples (random order within each laboratory is 
often best); (4) details regarding the reporting of study results 
on the reporting form; and (5) the time limit for return of the 
reporting form. 

9.4.1 Laboratories shall be required to report all figures 
obtained in making measurements, instead of rounding results 
before recording them. This may result in recording one or 
more significant figures beyond what may be usual in the 
Report section of the test method. A decision about rounding 
all data can be made by the task group when the final statistical 
analyses are performed. 

9.4.2 The laboratories shall report results from analyses of 
study samples without background subtraction and shall also 
report background levels for every matrix that they use in the 
study. The task group will make any background corrections 
that may be necessary. 

9.4.3 Zeros and negative numbers should be reported when¬ 
ever they represent the actual test results produced. Test results 
should never be censored by a participant. The reporting of less 
than or greater than results negates the objectivity of subse¬ 
quent statistical calculations and should be avoided. Never 
report zero in place of a less-than or other nonquantitative test 
result. 

9.5 The task group chair (or designee) should monitor the 
collaborative study to assure that results are reported back 
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wi thin the agreed upon time limit and are free of obvious 
procedural, transcription, clerical, or calculation errors. Careful 
design of the reporting form (or reporting instructions) will 
facilitate this task. 

10. Collaborative Study Data Analysis 

10.1 For each matrix/analyte, the steps involved by the task 
group chair in the data analysis consist of: ( 1) tabulating the 
data; (2) eliminating any laboratories that did not follow 
significant study instructions, were not in control during the 
study, or were so consistently high or low that their results are 
unreasonable (see 10.3); (3) eliminating any individual outlier 
data points (10.4); ( 4) for each matrix and analyte concentra¬ 
tion studied, calculating the overall and single-operator stan¬ 
dard deviations and means from the retained data and calcu¬ 
lating the bias from each mean spike recovery (must subtract 
the mean reported background value whenever necessary); ( 5) 
tabulating the statistics; (6) assembling information required 
for the research report; and, if desired, ( 7) summarizing these 
results in a graph or regression equation for the test method 
statement. 

10.1.1 As an aid to following the steps, the task group chair 
may find it helpful to review the sample calculations of 
precision and bias given in Appendix X3. 

10.2 Tabulation of Data —The data reported by the labora¬ 
tories shall be made consistent in reporting units and, if 
possible, in the number of reported values per operator or 
laboratory (10). Before beginning, remove any unusable data 
sets generated by laboratories that did not follow significant 
study instructions or used an unacceptable variation of the test 
method being studied. Unless each laboratory used its own 
matrix with a unique background concentration, all outlier 
testing and precision estimates are to be based on the concen¬ 
tration reported rather than on background-corrected results. 

10.2.1 Sometimes looking at the histogram of a set of data 
will help one recognize or understand, or both, the cause of 
unusual data. 

10.3 Rejection of Outlier Laboratories — If one or more 
laboratory’s data for an analyte in a specific matrix are so 
consistently high or low that there must be a large systematic 
error specific to that laboratory, all the data from the laboratory 
for that analyte/matrix should be rejected. Identify outlier 
laboratories by applying the Youden laboratory ranking test 
(11) at the 5 % significance level. 

10.3.1 For example, say n laboratories reported results for a 
specific matrix and analyte. Within the data set reported for 
each concentration, assign a rank score from 1 for the highest 
result to n for the lowest result. 

10.3.1.1 For this test, all n rank scores for each concentra¬ 
tion shall be assigned, even if one or more of the laboratories 
did not report a result for this particular concentration. The 
rank of any missing results should be the mean rank of the 
actual data reported by that laboratory for the other concentra¬ 
tions of the same analyte and matrix. Also, assign an appro¬ 
priate rank to nonquantitative results. 

10.3.1.2 Identical results would:each be given the average 
of the ranks the group is entitled to receive. 

10.3.2 For the matrix/analyte, total the rank scores for each 
laboratory over all of the q_ concentrations. If the total rank sum 
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for any particular laboratory is designated as R, then if either: 

R < the lower value in Table 1, or 
R > the upper value in Table 1, 

that laboratory is a candidate to be marked as an outlier and 
ignored in subsequent calculations with a 5% risk of this 
judgement being incorrect. 

10.3.2.1 If more than 20% of the laboratories reporting 
usable data for the matrix/analyte are outlier candidates, order 
the candidate laboratories according to the difference between 
their total rank sum and the nearest critical value given above, 
and reject individual or tied groups of laboratories until 
rejection of the next laboratory would exceed the 20 % limit. If 
rejection of a group of laboratories with equal distances would 
cause the 20 % limit to be exceeded, randomly reject labora¬ 
tories from the group until rejection of the next laboratory 
would exceed the 20 % limit. Data from laboratories ultimately 
marked as outliers should be ignored for subsequent calcula¬ 
tions. 

10.3.3 Repeat 10.3 for every matrix and analyte studied. 

10.4 Rejection of Unusable Data and Individual Outlier 
Results, and Calculation of Final Mean and Overall Standard 
Deviation Estimates : 

10.4.1 Reject nonquantitative responses since they are use¬ 
less for subsequent calculations. These rejections do not count 
against the 10 % limit in 10.4.4 because such responses are 
unusable. It is the task group’s responsibility to judge whether 
reported zeros are truly quantitative analytical results, and this 
should usually be done after consulting with each laboratory 
that reported a zero, whenever that is possible. 

10.4.2 Let the remaining data reported for a specific matrix/ 
analyte/concentration be designated x i9 i = 1 to n. Then 
calculate the mean (x) and overall standard deviation (s r ) as 
follows: 



and 

/£ (*,•-*) 2 

St= y— n-r - (2) 

10.4.3 Calculate the T value for the most extreme remaining 
value (x e ) as follows: 

T=(x e -x)/s T ( 3 ) 

If the absolute value of T is greater than the critical value for 
n measurements from Table 2, x e is considered an outlier value 
and ignored for subsequent calculations (12,13,14). 

10.4.4 If an outlier was just removed in 10.4.3, return to 

10.4.2 unless the removal of one more individual outlier would 
exceed 10% of the usable data originally reported for this 
matrix, analyte, and concentration. If 10.4.2 cannot be repeated 
for this matrix/analyte/concentration, proceed to the next step. 

10.4.5 Return to 10.4.2 for the next matrix, analyte and 
concentration, until final retained data sets and the related 
mean and overall standard deviation estimates are available for 
every combination studied. 
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TABLE 1 Upper and Lower Limits of the Acceptable Ranges for Total Rank Sums (5 % Level of Significance) 


Note 1—This table was prepared by James Longbottom, USEPA, NERL, Cincinnati, OH, and is an adaptation and extension of Youden’s Table 7 (3). 
According to Thompson and Willke (15), lower values in this table = g + n (0.05(g\)/2 ri) Vg - (g+ l)/2, 

and upper values — ng - n (0.05( g\)/2n) 1/g + (g + l)/2, where n = the number of laboratories and g = the number of concentrations. 


Number of 
Laboratories 





Number of Concentrations 





6 

8 


10 

12 

14 

Lower 

Upper 

Lower 

Upper 

Lower 

Upper 

Lower 

Upper 

Lower 

Upper 

7 

11 

37 

17 

47 

23 

57 

29 

67 

35 

77 

8 

12 

42 

18.5 

53.5 

25 

65 

32 

76 

39 

87 

9 

13 

47 

20 

60 

27.5 

72.5 

35 

85 

42.5 

97.5 

10 

14 

52 

21.5 

66.5 

29.5 

80.5 

38 

94 

46 

108 

11 

14.5 

57.5 

23 

73 

32 

88 

41 

103 

50 

118 

12 

15.5 

62.5 

24.5 

79.5 

34 

96 

43.5 

112.5 

53.5 

128.5 

13 

16.5 

67.5 

26 

86 

36.5 

103.5 

46.5 

121.5 

57 

139 

14 

17.5 

72.5 

27.5 

92.5 

38.5 

111.5 

49.5 

130.5 

60.5 

149.5 

15 

18 

78 

29 

99 

40.5 

119.5 

52.5 

139.5 

64 

160 

16 

19 

83 

30.5 

105.5 

42.5 

127.5 

55 

149 

67.5 

170.5 

17 

20 

88 

32 

112 

45 

135 

58 

158 

71.5 

180.5 

18 

21 

93.5 

33.5 

118.5 

47 

143 

61 

167 

75 

191 

19 

21.5 

98.5 

35 

125 

49 

151 

63.5 

176.5 

78.5 

201.5 

20 

22.5 

103.5 

36.5 

131.5 

51 

159 

66.5 

185.5 

82 

212 

21 

23 

109 

38 

138 

53.5 

166.5 

69 

195 

85 

223 

22 

24 

114 

39 

145 

55.5 

174.5 

72 

204 

88.5 

233.5 

23 

25 

119 

40.5 

151.5 

57.5 

182.5 

74.5 

213.5 

92 

244 

24 

25.5 

124.5 

42 

158 

59.5 

190.5 

77.5 

222.5 

95.5 

254.5 

25 

26.5 

129.5 

43.5 

164.5 

61.5 

198.5 

80 

232 

99 

265 

26 

27 

135 

45 

171 

63.5 

206.5 

83 

241 

102.5 

275.5 

27 

28 

140 

46 

178 

65.5 

214.5 

85.5 

250.5 

106 

286 

28 

29 

145 

47.5 

184.5 

67.5 

222.5 

88 

260 

109.5 

296.5 

29 

29.5 

150.5 

49 

191 

69.5 

230.5 

91 

269 

112.5 

307.5 

30 

30.5 

155.5 

50.5 

197.5 

71.5 

238.5 

93.5 

278.5 

116 

318 

31 

31 

161 

51.5 

204.5 

73.5 

246.5 

96.5 

287.5 

119.5 

328.5 

32 

32 

166 

53 

211 

75.5 

254.5 

99 

297 

123 

339 

33 

32.5 

171,5 

54.5 

217.5 

77.5 

262.5 

101.5 

306.5 

126 

350 

34 

33.5 

176.5 

55.5 

224.5 

79.5 

270.5 

104.5 

315.5 

129.5 

360.5 

35 

34 

182 

57 

231 

81.5 

278.5 

107 

325 

133 

371 

36 

35 

187 

58.5 

237.5 

83.5 

286.5 

109.5 

334.5 

136 

382 

37 

35.5 

192.5 

59.5 

244.5 

85.5 

294.5 

112.5 

343.5 

139.5 

392.5 

38 

36.5 

197.5 

61 

251 

87.5 

302.5 

115 

353 

143 

403 

39 

37 

203 

62.5 

257.5 

89.5 

310.5 

117.5 

362.5 

146 

414 

40 

38 

208 

63.5 

264.5 

91.5 

318.5 

120 

372 

149.5 

424.5 

41 

•' 38,5 

213.5 

65 

271 

93.5 

326.5 

123 

381 

153 

435 

42 

39 

219 

66 

278 

95.5 

334.5 

125.5 

390.5 

156 

446 

43 

40 

r 224 

67.5 

284.5 

97 

343 

128 

400 

159.5 

456.5 

44 

40.5 

229.5 

69 

291 

99 

351 

130.5 

409.5 

162.5 

467.5 

45 

41,5 .; 

234,5 

70 

298 

101 

359 

133 

419 

166 

478 

46 

42 

240 

71.5 

304.5 

103 

367 

136 

428 

169.5 

488.5 

47 

43 

245 

72.5 

311.5 

105 

375 

138.5 

437.5 

172.5 

499.5 

48 

43.5 

250.5 

74 

318 

107 

383 

141 

447 

176 

510 

49 J.-:.. 

44 

256 

75.5 

324.5 

108.5 

391.5 

143.5 

456.5 

179 

521 

50 

45 

261 

76.5 

331.5 

110.5 

399.5 

146 

466 

182.5 

531.5 


■j. 


10.5 Calculation of Single-Operator Standard Deviation 
Estimates : 

10.5.1 To complete the required statistical calculations, 
estimate the single-operator standard deviation (s Q ) from the 
retained data pairs available for each Youden pair, analyte, and 
matrix in the study as follows: 



yj‘- 


(A - of 


2 (m — 1 ) 


(4) 


where: 

m = the number of retained pairs of results available for 
that Youden pair, analyte, and matrix, 

D i = the difference between the retained value from labo¬ 
ratory i for the Youden sample with the higher true 
value of the pair minus the retained value from 
laboratory i for the other sample of the pair, and 
D = the mean of the m usable D t values. 

10.5.2 The calculation of s Q for a blind duplicate is the same 
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TABLE 2 Critical Values for T {Two-Sided Test at a 5 % 
Significance Level) When Standard Deviation is Calculated from 
the Same Samples (for Single-Value Outlier Testing) {see 10.4)^ 


Number of Useable Values, n 

Critical Value for T 

7 

2.02 

8 

2.13 

9 

2.21 

10 

2.29 

11 

2.36 

12 

2.41 

13 

2.46 

14 

2.51 

15 

2.55 

16 

2.58 

17 

2.62 

18 

2.65 

19 

2.68 

20 

2.71 

21 

2.73 

22 

2.76 

23 

2.78 

24 

2.80 

25 

2.82 

30 

2.91 

35 

2.98 

40 

3.04 

45 

3.08 

50 

3.13 

60 

3.20 

70 

3.26 

80 !' . • 

3.30 

90 

3.35 

100 

3.38 


A Values of T for n s 25 are based on Grubbs (12). For n > 25, the values of T 
are approximate. All values have been adjusted for division by n -1 instead of n 
in calculating Tabulated values come from Practice E 178 and may also be found 
in Grubbs (14), although the level of significance shown in Practice E 178 has 
been doubled because our use here is a two-sided test, rather than a one-sided 
test. . ^ ~ ; ; 


as for a Youden pair. One of the duplicate samples is arbitrarily 
selected as the higher sample for this calculation. _ 

10.6 Calculation of Bias: 

10.6.1 The calculation of the bias of a test method will 

logically follow the collaborative study design (7.4). The usual 
collaborative study technique will involve reporting the recov¬ 
ery of added (therefore known) amounts of the analytes being 
measured. c '-’~ .- 

10.6.2 The calculation of bias for a specific matrix, analyte, 
and concentration is as follows: 

Bias (%) - 100(jc - b - c) / c (5) 

where: . 

x = the mean of retained data for that matrix, analyte and 
concentration, ; 

c = the true concentration added, and 
b = the mean background concentration reported, if nec¬ 
essary. .. , ; 

10.6.3 Where other types of studies are used to develop a 
true concentration for use in estimation of the test method bias, 
special care shall be taken to assure that the other study 
provides a logical reference value. Consultation with the 
Results Advisor and other recognized experts may be appro¬ 
priate in such cases. 


11. Format of the Precision and Bias Statement Required 
in Each Test Method 

1-1.1 For most test methods, a collaborative study will be 
conducted and the following requirements apply. 

11.1.1 A brief note shall provide the reader of the test 
method with a complete understanding of the collaborative 
study conducted. At a minimum, this note shall include the 
number of laboratories that contributed data, the matrices 
studied, the version of Practice D 2777 followed in designing 
and analyzing the study data, and any other significant aspects 
of the study not presented elsewhere in the test method. 

11.1.1.1 Regarding significant study aspects that must be 
described, if the analytical conditions used during the collabo¬ 
rative study were more restrictive than those allowed in the test 
method, it is particularly important that these restrictive con¬ 
ditions be fully described in the Precision and Bias statement of 
the test method. Results from the collaborative study may not 
apply to other analytical conditions allowed in the test method. 

11.1.2 The following caution shall also be included, “Re¬ 
sults of this collaborative study may not be typical of results for 
matrices other than those studied ” 

11.1.3 The study results shall always be available in the 
form of a table, which, for each matrix, analyte, and concen¬ 
tration studied, will usually include the true concentration (c) 
added to the matrix, and must include the number of values 
reported, the number of values retained (that is, left after outlier 
testing), and (from the retained data): ( 1 ) the mean response ( 
A), (2) bias as a percent of c, and ( 3 ) the overall standard 
deviation ( s T ). For each matrix, analyte, and Youden pair of 
sample concentrations, the table shall include the number of 
retained data pairs and the single-operator standard deviation 
(s^) estimated from these pairs of retained values. This table 
shall be included in the test method unless equivalent math¬ 
ematical or graphical relationships of the mean (or bias), s T and 
s 0 , to concentration are provided instead. If a matrix had a 
naturally occurring, non-zero background level for this analyte, 
the mean background level reported by laboratories passing the 
outlier testing for the Youden pair with the lowest study 
concentration shall also be reported in this table, and the bias 
estimates shall be calculated from the recovery of the true 
spikes, that is, x—average background. This table shall always 
be included in the research report provided to the Results 
Advisor and filed at ASTM Headquarters. If the full table is not 
included in the test method, at least a listing of the true 
concentrations studied for each matrix and analyte, and the 
number of values retained for each, shall be included in the 
precision and bias statement. 

11.1.4 Mathematical or graphical relationships developed 
from the study results shall represent the general way precision 
and bias vary with concentration. These relationships can be 
very helpfiil to a user of a test method who must estimate the 
precision and bias at a specific concentration within the range 
studied. Graphs that simply connect the estimates from the 
collaborative study (connect the dots) are not acceptable. 
Mathematical relationships shall be accompanied by some 
indication of the goodness of their fit to the study statistics, 
unless those statistics are given in the test method. 

11.2 If there is some reason why a full collaborative study 
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could not be done, the precision and bias statement shall 
present a complete justification with reference to 1.3, 1.4, or 
1.5, whenever appropriate. If a special exemption was ap¬ 
proved by Committee D-19 on the recommendation of the 
Results Advisor and the Technical Operations Section of the 
Executive Subcommittee of Committee D-19, the date of that 
exemption shall also be provided. 

11.3 Test Methods with Non-Numerical Reports : 

11.3.1 When a method specifies that a test result is a 
non-numerical report of success or failure based on criteria in 
the procedure, the statement on precision and bias should read 
as follows: 

11.3.1.1 Precision and Bias —No statement is made about 
either the precision or the bias of Method DXXXX for 
measuring (insert here the name of property) since the result 
merely states whether there is conformance to the criteria for 
success specified in the procedure. 

11.4 Test Methods Specifying Other Procedures : 

11.4.1 When a method specifies that the procedures in 
another ASTM method are to be used, a statement such as the 
following should be used to assure the user that precision and 
bias statements apply. 

11.4.1.1 Precision and Bias —The precision and bias of this 
test method of measuring (insert here the name of the property) 
are as specified in Method (insert here the designation of the 
other method). 

12. Approval of Data Analysis and Statements 

12.1 Approval of the precision and bias statement shall be 
obtained from the Results Advisor before the test method is 
submitted for committee ballot, providing him/her with a copy 
of: 

12.1.1 All test data resulting from the collaborative test. 


12.1.2 All statistical calculations. 

12.1.3 A summary of the final statistical estimates in tabular 
form. 

12.1.4 A copy of the final test method, including the 
precision and bias statement based on the study results. 

12.1.5 A copy of every document given to the participants 
during the collaborative study. 

12.1.6 A complete list of the laboratories (names, addresses, 
principal contact, etc.) that participated in the study. Do not 
identity the source of specific study data using anything other 
than randomly assigned laboratory numbers or codes. The 
relationship between these numbers/codes and the contributing 
laboratories must be held strictly confidential. 

12.1.7 A description of how the study samples were pre¬ 
pared, etc. 

12.1.8 Any background information that may have influ¬ 
enced the results and any other information required for the 
research report, along with a copy of correspondence docu¬ 
menting approval by the Results Advisor. 

12.1.9 Once satisfied with this study file, the Results Advi¬ 
sor shall see that it is sent to ASTM for filing as the official 
research report. 

12.2 Experimental Data —The precision and bias statement 
in the test method shall include a footnote indicating where the 
supporting data can be found. The footnote shall read as in the 
following example: 

Supporting data for the precision and bias statements have 
been filed at ASTM Headquarters. Request RR:D_. 

13. Keywords 

13.1 collaborative study; interlaboratory study; method 
bias; method precision; method recovery; round-robin study; 
statistical analysis; Youden study design 


APPENDIXES 
(Nonmandatory Information) 

XI. APPROVAL OF STUDY DESIGN 

Xl.l Using Test Method D 5790 also known as USEPA preparation of the samples for the interlaboratory study actu- 

Method 524.2, as an example, Fig. Xl.l was sent by the Task ally began. 

Group Chair to the Results Advisor for his approval before 
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TO: 

PROM: 


D-19 Results Advisor 


Task Group Chairman 


_____ th^Aii 

Date 

The following details for a proposed collaborative study are 
respectfully submitted for your review and approval: 

X. 1.1 Test method title (inc. draft number and date); 



Copy 
see ATTACHMENT # 


format and as approved by Task Group, 


X.l.2 Analyte(s): ^ 3 

X.1.3 Procedure for estimating bias: - f 

jI^juUk hJbte*j~Uvg , 


X.1.4 Number of Youden sample pairs 


(Minimum of 3 req._) 


X.l.5 Intended operating range of test method: &• l &O __ 

and approx, mean concentration of each sample pair in study; 

- O.g, L , £ , &«?, 1 £. --- 

Units (spelled out): . . . 

X. 1.6 Estimated number of laboratories ~73 and analysts 7 3 
(Should be > 8 labs to guarantee 6 values after outlier testing.) 

X.1.7 Matrices being tested (at least 1 reproducible matrix)^ _ 

_iz rr zrz_' 

X.1.8 QC Samples being tested : unflE tUul^ 

X.1.9 Plans for developing study samples: ATTACHMENT # ^ 

X.1.10 Participant's instruction package: ATTACHMENT # 3 

X.1.11 Participant's data reporting form: ATTACHMENT #, 


4 - 


Approved by 


D-19 Results Advisor 


Date 


FIG. XI .1 Approval of Study Design: Form A—Approval of Plans for Interlaboratory Testing 


X2. REPORTING OF STUDY DATA 


X2.1 An example of the data reporting forms that could 
have been submitted by each participating laboratory for each 
analyte is provided as Fig. X2.1. 

X2.2 Each participant was also required to provide specific 
information defining their analytical system and the analytical 


conditions they used from among options allowed in the test 
method. On this questionnaire, they were also encouraged to 
provide any comments they considered appropriate. 
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Complete Title of Test Method, Draft Number, and Date; Measurement of Purgeable Organic 
Compounds in Water by Capillary Column Gas Chromatography, Draft 3, July 18,1991. 


Laboratory Code: 1 
Analyte; Chlorobenzene 


Reagent Water 

Drinking Water 

Wastewater 

Ground Water 

TCLP Buffer 

Sple 

No. 

Result 

Hg/L 

Sple 

No. 

Result 

Mg/L 

Sple 

No. 

Result 

Mg/L 

Sple 

No. 

Result 

Mg/L 

Sple 

No. 

Result 

Mg/L 

QC1 

9.06 

QC2 


QC3 


QC4 

9.98 

QC5 


Bkg 

nd 

Bkg 


Bkg 


Bkg 

nd 

Bkg 


1 

0.23 

11 


21 


31 

nd 

41 


2 

0.21 

12 


22 


32 

0.20 

42 


3 

1.24 

13 


23 


33 

1.14 

43 


4 

23.82 

14 


24 


34 

21.95 

44 


5 

1.08 

15 


25 


35 

0.97 

45 


6 

5.71 

16 


26 


36 

5.25 

46 


7 

19.21 

17 


27 


37 

18.04 

47 


8 

4.45 

18 


28 


38 

4.61 

48 


9 

82.99 

19 


29 


39 

77.89 

49 


10 

67.65 

20 


30 


40 

65.23 

50 



FIG. X2.1 Reporting of Study Data: Sample of Form B—Data Report from Individual Laboratories 


X3, SAMPLE CALCULATION OF PRECISION AND BIAS 


X3.1 The following is a sample of the precision and bias 
calculations from the data.reported in the Test Method D 5790 
study for one analyte in one matrix. These procedures shall be 
followed for each analyte and matrix combination in the study. 

X3.2 Example data are presented in Table X3.1, as sug¬ 


gested in 10.2. Note that values shown represent analytical 
results after correction for background concentration by the 
task group or its representative, the study coordinator. 

X3.3 Test for lab-ranking outliers (see 10.3). Table X3.2 
shows the results of the lab-ranking calculations on the data in 


TABLE X3.1 ASTM Test Method D 5790: Reagent Water, 5 mL Purge—Raw Data for Chlorobenzene Analysis A 


Laboratory 
or Analyst 

; 



Concentration in pg/L 




Sample 5 P 
0.88 

Sample 3 

1.10 

Sample 8 

4.41 

Sample 6 

5.29 

Sample 7 
17.64 

Sample 4 
22.05 

Sample 10 
61.73 

Sample 9 
74.96 

1 

1.08 

1.24 

4.45 

5.71 

19.21 

23.82 

67.65 

82.99 

6 

2,35 

0.96 

4.53 

5.24 

17.14 

21.43 

64.30 

70.40 

8 

1.30 

1.30 

4.90 

6.80 

21.70 

25.60 

61.40 

85.40 

15 

1.20 

1.40 

3.90 

4.80 

15.70 

18.70 

54.10 

66.10 

21 

2.20 

0.93 

4.90 

4.00 

16.90 

18.10 

53.80 

81.80 

25 

1.21 

1.10 

4.50 

5.37 

17.90 

22.22 

62.10 

75.10 

26 

1.20 

1.20 

4.40 

4.90 

16.70 

21.50 

62.40 

71.80 

27 

1.10 

1.00 

4.30 

5.80 

22.10 

26.60 

75.00 

89.10 

31 

0.80 

0.00 

5,30 

5.50 

19.10 

24.03 

74.80 

88.90 

38 

1.30 

1.70 

4.70 

6.60 

23.50 

24.10 

74.40 

89.50 

47 

1.10 

1.20 

4.10 

5.30 

17.90 

22.40 

77.90 

63.50 

49 

1.00 

1.30 

4.90 

5.40 

12.80 

18.70 

26.10 

37.60 

52 

1.20 

1.10 

4.80 

5.60 

19.80 

23.50 

69.80 

83.10 

54 

0.55 

0.79 

3.33 

3.65 

14.31 

17.86 

50.41 

60.89 

56 

1.00 

1.30 

4.70 

5.80 

19.30 

24.10 

66.50 

82.90 


A Values represent analytical results after correction for background concentration by the study coordinator. 
B Change to match sample identification used during study. 
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TABLE X3.2 ASTM Test Method D 5790: Reagent Water, 5 mL Purge—Ranking Test for Chlorobenzene Analyses 


Laboratory 
or Analyst 

Sample 5 A 

Sample 3 

Sample 8 

Sample 6 

Sample 7 

Sample 4 

Sample 10 

Sample 9 

Rank Sum by 
Laboratory 
or Analyst 

1 

11 

6 

10 

5 

6 

6 

6 

6 

56 

6 

1 

12 

8 

11 

10 

11 

8 

11 

72 

8 

3.5 

4 

3 

1 

3 

2 

11 

4 

31.5 

15 

7 

2 

14 

13 

13 

12.5 

12 

/ 12 

85.5 

21 

2 

13 

3 

14 

11 

14 

13 

8 

78 

25 

5 

9.5 

9 

9 

8.5 

9 

10 

9 

69 

26 

7 

7.5 

11 

12 

12 

10 

9 

10 

78.5 

27 

9.5 

11 

12 

3.5 

2 

1 

2 

2 

43 

31 

14 

15 

1 

7 

7 

5 

3 

3 

55 

38 

3.5 

1 

6.5 

2 

1 

3.5 

4 

1 

22.5 B 

47 

9.5 

7.5 

13 

10 

8.5 

8 

1 

13 

70.5 

49 

12.5 

4 

3 

8 

15 

12.5 

15 

15 

85 

52 

7 

9.5 

5 

6 

4 

7 

5 

5 

48.5 

54 

15 

14 

15 

15 

14 

15 

14 

14 

116 c 

56 

12.5 

4 

6.5 

3.5 

5 

3.5 

7 

7 

49 


A Change to match sample identification used during study. 

B This rank sum is below the lower limit given in Table 1; reject all data from this laboratory for this analyte. 
c This rank sum is above the upper limit given in Table 1; reject all data from this laboratory for this analyte. 


Table X3.1. Since 20 % of the 15 laboratories reporting usable 
data is exactly three, up to three laboratories can be removed 
with this test. Laboratory 38 fails for providing consistently 
high responses and Laboratory 54 fails for providing consis¬ 
tently low responses, relative to the other laboratories that 
reported. 

X3.4 There are no less-than values to reject as unusable; 
however, the zero reported by Laboratory 31 for Sample 3 is 
not considered to be a legitimate quantitative response and is 
therefore rejected as unusable. Under normal study conditions, 
Laboratory 31 would be contacted to resolve questions regard¬ 
ing their zero response, but this was not possible for prepara¬ 
tion of this example. 

X3.5 Calculate the initial mean ( XBAR ) and standard 
deviation (sy) of the remaining data for each concentration and 
calculate the initial single-outlier test values, T (see 10.4). 
Since at least one value, but not more than 10 % of the usable 
data, can be removed for each concentration, and 10 % of 13 is 
less than two, at most, one value can be removed using 


single-outlier testing, for each concentration. Only the T values 
for the most extreme values for Samples 10 and 9 exceed the 
2,46 critical value for sets of 13 values, and so are removed as 
outliers. Table X3.3 gives the results of these calculations. 

X3.6 Table X3.4 shows the data with all outliers indicated 
and Table X3.5 contains the final statistics. 

X3.7 From the final statistics, the responsible task group 
chose to develop the following regressions to relate XBAR , s B 
and s 0 to the true concentration ( C) for C values between 0.88 
and 75 micrograms per litre: 


XBAR = 1.035 (C) + 0.03, ( R 2 = 1.00) 
s T = 0.119 (C) + 0.01, (R 2 = 0.97) 

.s 0 = 0.074 (Q + 0.08, (R 2 = 0.75) 

R 2 indicates the proportion of the total variability in the 
dependent variable which can be explained by the regression. 

Note X3.1—This X3.7 step is optional and need not be followed by 
other task groups. 


TABLE X3.3 ASTM Test Method D 5790: Reagent Water, 5 mL Purge—Single-Outlier Tests for Chlorobenzene Analyses 


Sample Number 

5 

3 

8 

6 

7 

4 

10 

9 

True concentration (Q. pg/L 

0.88 

1.10 

4.41 

5.29 

17.64 

22.05 

61.73 

74.96 

Number of retained values 

13 

12 

13 

13 

13 

13 

13 

13 

Mean recovery (XBAR ) w ■ ; 

1.29 

1.17 

4.59 

5.40 

18.17 

22.36 

62.76 

75.28 

Overall standard deviation ( s T ) 

0.46 

0.15 

0.38 

0.65 

2.48 

2.65 

13.28 

14.08 

Most extreme value 

2.35 

0.93 

5.30 

4.00 

12.80 

18.10 

26.10 

37.60 

Single-outlier test value (7) 

2.30 

1.60 

1.87 

2.15 

2.17 

1.61 

2.76 A 

2.68 ^ 


A This T value exceeds the critical value of 2.46 in Table 2 for sets of 13 reported values. 
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TABLE X3.4 ASTM Test Method D 5790: Reagent Water, 5 mL Purge—Retained Data for Chlorobenzene Analyses 

Note 1* —Current significance levels: 

1. Lab ranking data rejection tests, alpha = 0.05, 

2. Individual outlier tests using Thompson(s) role, alpha = 0,05. 


Concentration in |jg/L 


Laboratory 
or Analyst 

Lab 

Rejected 

Sample 5 
0.88 

Sample 3 

1.10 

Sample 8 
4.41 

Sample 6 
5.29 

Sample 7 
17.64 

Sample 4 
. 22.05 .... 

Sample 10 
> 61.73 

Sample 9 
74.96 

1 


1.08 

1.24 

4.45 

5.71 

19.21 

23.82 

67.65 

82.99 

6 


2.35 

0.96 

4.53 

5.24 

17.14 

21.43 

64.30 

70.40 

8 


1.30 

1.30 

4.90 

6.80 

21.70 

25.60 

61.40 

85.40 

15 


1.20 

1.40 

3.90 

4.80 

15.70 

18.70 

54.10 

66.10 

21 


2.20 

0.93 

4.90 

4.00 

16.90 

18.10 

53.80 

81.80 

25 


1.21 

1.10 

4.50 

5.37 

17.90 

22.22 

62.10 

75.10 

26 


1.20 

1.20 

4.40 

4.90 

16.70 

21.50 

62.40 

71.80 

27 


1.10 

1.00 

4.30 

5.80 

22.10 

26.60 

75.00 

89.10 

31 


0.80 

0.00* 

5.30 

5.50 

19.10 

24.03 

74.80 

88.90 

38 

B 

1.30 1 2 3 4 5 6 7 8 

1.70 s 

4.70 s 

6.60 s 

23.50 8 

24.10 s 

74.40 8 

89.50 8 

47 


1.10 

1.20 

4.10 

5.30 

17.90 

22.40 

77.90 

63.50 

49 


1.00 

1.30 

4.90 

5.40 

12.80 

18.70 

26.10 s 

37.60 8 

52 


1.20 

1.10 

4.80 

5.60 

19.80 

23.50 

69.80 

83.10 

54 

B 

0.55 s 

0.79 s 

3.33 s 

3.65 s 

14.31 s 

17.86 s 

50.41 8 

60.89 8 

56 


1.00 

1.30 

4.70 

5.80 

19.30 

24.10 

66.50 

82.90 


* = Rejected as a nonquantitative response. 
8 = Rejected. 


TABLE X3.5 ASTM Test Method D 5790: Reagent Water, 5 mL Purge—Final Statistical Summary for Chlorobenzene Analyses 


Sample Number 

5 

3 

8 

6 

7 

4 

10 

9 

Number of retained values 

13 

12 

13 

13 

13 

13 

12 

12 

True concentration (C) pg/L 

0.88 

1.10 

4.41 

5.29 

17.64 

22.05 

61.73 

74.96 

Mean recovery ( XBAR) 

1.29 

1.17 

4.59 

5.40 

18.17 

22.36 

65.81 

78.42 

Percent recovery 

146.33 

106.29 

104.10 

102.11 

103.02 

101.41 

106.61 

104.62 

Overall standard deviation (Sy) 

0.46 

0.15 

0.38 

0.65 

2.48 

2.65 

7.74 

8.74 

Overall relative standard deviation,% 

35.50 

12.91 

8.24 

11.99 

13.64 

11.85 

11.77 

11.15 

Number of retained pairs 

12 


13 


13 


12 


Single standard deviation, (s 0 ) 

0.40 


0.48 


0.80 


7.31 


Analyst relative deviation,% 

32.60 


9.68 


3.94 


10.14 
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with any item mentioned in this standard. Users of this standard are expressly advised that determination of the validity of any such 
patent rights, and the risk of infringement of such rights, are entirely their own responsibility. 
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